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Executive Summary 


This paper presents the results of a small-scale validity study examining 
relationships between new SAT Essay scores and first-semester grades in 
English and writing courses, as well as first-year grade point average (FYGPA) in 
college. The sample includes more than 4,500 first-year students across 10 four- 
year institutions. Results show that there is a positive relationship between the 
three SAT Essay score dimensions and both first-semester English and writing 
course grades and FYGPA. These findings provide support for institutions using 
or considering the use of SAT Essay scores for admission, placement, or 
advising. 
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Introduction 


The Importance of the SAT Essay 


College professors, administrators, admission officers, and other higher education leaders 
expect students to arrive at college ready and able to write, as it is among the most critical 
skills for college success (Beyer, Gillmore, & Fisher, 2007; Conley, 2007). Knowing when 
writing is a weakness for a student and when additional instruction or support is needed is 
essential for advancing students on the path to success in college (Kuh, 2006). Research 
has also shown that the amount of writing a student does for a course is linked toa 
student’s level of engagement with that course and has a positive impact on measures of 
critical thinking (Light, 2001; Pascarella & Terenzini, 2005). 


High school English and writing experiences vary widely, and high school grades are not 
always the best measure of a student’s writing ability (Conley, 2007). High school grades 
understandably include a lack of consistency with regard to scales and standards across 
high schools and teachers, are based on different types of assignments with different levels 
of rigor, and can include subjective information about the student unrelated to the content 
area being evaluated (Shaw, 2018). The SAT® Essay, which was implemented in March 
2016, was developed in consultation with higher education faculty, in part, to fill the need to 
thoughtfully and fairly measure and understand student writing performance in an authentic, 
consistent, and nuanced way. It reflects college expectations of students by requiring them 
to analyze a text, formulate logical arguments, and appropriately use evidence in their 
writing (College Board, in preparation). 


Essay testing has advantages over simply using multiple-choice formats, as essays focus on 
more global aspects of writing (Matzen & Hoyt, 2004; Shaw & Kobrin, 2012). Essays also 
measure skills (e.g., analysis of persuasive techniques) that are not as easily measured in 
multiple-choice formats and do not have the same conceptual organization of being either 
“correct” or “incorrect” the way multiple-choice questions do (Stiggins, 1982). 


There are also advantages to having students share a writing sample produced in real time, 
independent of the work or input of others, and subject to test security measures for 
admission purposes. This is in contrast to other written materials for the college application 
such as essays or information on awards, honors, or activities, which can be difficult to verify 
for authenticity (Laird, 2005). The SAT Essay provides a way to evaluate all students’ writing 
skills and readiness in a fair and standardized way within an assessment that focuses on the 
type of analytic writing that students will be doing in college. 


Getting to Know the SAT Essay 


The SAT Essay is closely aligned with the analytical writing required of students in college. It 
assesses whether students can comprehend an appropriately challenging source text and 
prepare a written analysis that uses critical reasoning skills and evidence from the source 


4 Oo CollegeBoard 


text. It is different from many standardized direct-writing assessments as many other writing 
assessments ask students to write a persuasive essay based on their opinion. 


The SAT Essay was developed to have a common prompt and SAT form—specific source 
texts. The same prompt is presented to students in every test. This has significant 
advantages for students compared to how most essay tests are administered. Students are 
able to focus their preparation on developing important reading, analysis, and writing 
knowledge and skills instead of trying to guess what question will be asked on test day. 
Students can get right to work instead of spending valuable time trying to form an opinion on 
a topic they might not have spent much time thinking about. You can find the prompt below: 


As you read the passage below, consider how [the author] uses evidence, such as 
facts or examples, to support claims. 


e evidence, such as facts or examples, to support claims. 

e reasoning to develop ideas and to connect claims and evidence. 

e Stylistic or persuasive elements, such as word choice or appeals to emotion, 
to add power to the ideas expressed. 


Write an essay in which you explain how [the author] builds an argument to persuade 
[his/her] audience that [author's claim]. In your essay, analyze how [the author] uses 
one or more of the features listed above (or features of your own choice) to 
strengthen the logic and persuasiveness of [his/her] argument. Be sure that your 
analysis focuses on the most relevant features of the passage. Your essay should 
not explain whether you agree with [the author's] claims, but rather explain how the 
author builds an argument to persuade [his/her] audience. 


The SAT Essay source texts are passages that present arguments that examine ideas, 
debates, and trends in the arts, sciences, and civic, cultural, and political life. All source 
passages are taken from high-quality sources that have previously been published. The 
level of complexity of each passage is appropriate for high school students and is 
approximately 650—750 words in length. The SAT Essay is an optional component of the 
test and is administered at the end of the SAT. Test takers have 50 minutes to complete the 
essay. Refer to the Appendix for an example of the SAT Essay prompt and source text. 


SAT Essay scores are reported on three dimensions. The dimension scores are Reading, 
Analysis, and Writing, and they are each reported on a scale of 2-8. To perform well on the 
Reading dimension, a test taker must show effective comprehension of the source text, an 
understanding of the central idea(s) and important details, and must appropriately use 
evidence to demonstrate his or her understanding. To perform well on the Analysis 
dimension, a test taker must decide which features of the source text are most relevant to 
the persuasiveness of the text and then develop an effective explanation of how those 
features help the author build his or her argument. Finally, to perform well on the Writing 
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dimension, a test taker’s response must be cohesive and demonstrate effective use and 
control of language. 


Two independent and trained raters give a rating of 1-4 on each dimension, and these 
scores are combined to get the total score for each dimension. The majority of essay raters 
are current teachers or have previously taught high school or college-level courses that 
require writing, and all must have a bachelor’s degree or higher. As the essay Is optional, 
the scores are reported separately from other SAT scores and do not factor into the SAT 
total score or the SAT Evidence-Based Reading and Writing score. 


Common uses of the SAT Essay include using the scores in admission decisions, placing 
students in courses, and advising students. To collect validity evidence supporting the use 
of SAT Essay scores in admission, placement, or advisement processes, a first critical step 
is to understand the score relationships with relevant college outcomes. This study 
represents an initial examination of those relationships. 


Methods 


The current study explores the relationship between scores on the SAT Essay and grades in 
college. Specifically, first-semester English and writing course grades and first-year grade 
point averages (FYGPAs) will be examined for each of the three SAT Essay scores: 
Analysis, Reading, and Writing. 


Institution Sample 


Data for this study came from 10 diverse four-year institutions that varied in terms of size, 
selectivity, institutional control (public or private), and region of the U.S. (See Table 1). The 
sample of institutions was relatively equally divided in terms of control and region. Since the 
SAT Essay is optional and not all students take it, there were more selective institutions and 
more large institutions in the study sample to ensure a large enough sample of students with 
SAT Essay scores. 


Table 1: Institutional Characteristics 


Institutional Characteristics (k=10) 


Control Private 
Public 

Admittance Rate Under 50% 
50% to 75% 
Over 75% 


Undergraduate Enrollment Small 
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Medium 


Very Large 


U.S. Region Midwest 
Mid-Atlantic 
New England 
South 
Southwest 


West 


Note: Undergraduate enrollment was categorized as follows: small—750 to 1,999; medium—2,000 to 7,499; large—7,500 to 
14,999; and very large—15,000 or more. 


Student Sample 


The initial sample had 38,277 students and then was further cleaned and coded for final 
analysis. To be included in the current study, a student had to have taken the SAT Essay 
and also have a first-semester English and writing GPA. This resulted in a final sample size 
of 4,583 students. See Table 2 for sample characteristics by gender, race/ethnicity, and 
highest parental education level. Fully 60% of the sample was female, 62% of the sample 
was white, 85% of the sample noted their best language as English only, and 74% of the 
sample had at least one parent with a bachelor’s degree or higher. 


With regard to academic characteristics, the mean SAT Evidence-Based Reading and 
Writing score for the sample was 617 (SD=72), mean SAT Math score for the sample was 
611 (SD=82), and mean self-reported high school grade point average (HSGPA) was 3.77 
(SD=0.42). The demographic and academic sample characteristics suggest that the study 
sample is less diverse and more highly able than the full cohort of 2017 college-bound 
students (College Board, 2017), which was expected given that these students are enrolled 
in four-year institutions and had taken the optional SAT Essay that is required or 
recommended at more selective institutions. When a larger and more diverse sample of 
students is available for analysis, it is expected that the results of that study will show even 
clearer and more stable patterns and relationships resembling those in the current study. 


Table 2: Student Characteristics 


Student Characteristics (n=4,583) 


Gender 1,818 40% 


2,765 60% 
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Race/Ethnicity American Indian or Alaska Native 
Asian 
Black or African American 
Hispanic or Latino 
Native Hawaiian or Other Pacific Islander 
White 
Two or More Races 
Not Stated 
Best Language English Only 
English and Another 
English and Spanish 
Another Language 


Not Stated 


Highest Parental No High School Diploma 
Education Level 


High School Diploma 
Associate Degree 
Bachelor's Degree 
Graduate Degree 


Not Stated 


Note: Percentages may not sum to 100 due to rounding. 
Measures 


SAT Essay Scores. Data for the study were based on students’ most recent test scores. 
Each of the three SAT Essay scores is reported on a scale of 2-8. While a score of 0 is 
possible (and rare) when a student has an off-topic response, refuses to participate, 
provides either an illegible response or one not in English, or responds with an insufficient 
amount of text to score, students with a score of 0 were excluded from this study. The SAT 
Essay Reading score mean was 6 (SD=1.0), Analysis score mean was 5 (SD=1.3), and 
Writing score mean was 6 (SD=1.0). 


SAT Questionnaire Responses. Self-reported gender, race/ethnicity, best language, and 
parental education level were obtained from the SAT Questionnaire that students complete 
during registration for the SAT. 


English and Writing GPA (EWGPA). College coursework, provided by the participating 
institutions, was coded by content area. First-semester English and writing courses were 
then included in a calculation of the English and Writing GPA (EWGPA). The EWGPA was 
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calculated for each student, across all relevant numeric course grades received in English 
and writing courses during the first semester of college (excluding remedial coursework). 
Only coursework with valid numeric grades were included in the EWGPA. Valid nonnumeric 
grades such as P, |, or W were excluded. For example, if a student took only one English 
course in the first semester, then his or her average course grade in English/Writing would 
be based on the grade earned in that one course. If two English courses and one Writing 
course were taken, then the average course grade would be based on the average of those 
three course grades earned (taking into account the grades earned and the number of 
credits associated with each grade). EWGPA ranges from 0 to 4.00. In this sample the mean 
EWGPA was 3.47 (SD=0.69). Prior research has found that English grades tend to be 
among the highest grades earned in college (e.g., Marini, Shaw, Young, & Ewing, 2018). 


First-Year Grade Point Average (FYGPA). First-year grade point averages (FYGPA) 
based on grades in all courses through the first year were obtained from institutions 
participating in the national SAT Validity Study. FYGPA ranged from 0 to 4.00. In this 
sample the mean FYGPA was 3.23 (SD=0.62). 


Analyses 


The focus of the current study is on gathering early validity evidence for the use of the SAT 
Essay in college admission and placement decisions. Therefore, analyses are primarily 
descriptive and graphical in nature depicting the relationship between SAT Essay scores 
and each criterion of interest. The graphical relationship of each of the three SAT Essay 
scores (Reading, Analysis, and Writing) is shown with first-semester EWGPA and FYGPA. 


Results 


First-Semester English and Writing GPA (EWGPA) 


Figures 1 through 3 show the relationship between each SAT Essay score dimension and 
students’ first-semester EWGPA. All of the SAT Essay score dimensions show a positive 
relationship with first-semester EWGPA. There are slight deviations from this relationship 
seen at the lower and upper ends of the scale on a dimension, and this is likely due to fewer 
students and less stable sample sizes earning those score points in the study sample. 
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Figure 1 shows the average EWGPA earned by students at each Reading score point. This 
figure shows a positive relationship between the Reading score and first-semester EWGPA. 
In other words, there is a general trend of increasing average EWGPA as Reading scores 
increase. As an example from the graph, you can see that the average EWGPA for a 
student with a Reading score of 3 is a 3.05, while the average EWGPA for a student with a 
Reading score of 7 is a 3.64. 


Figure 1: Mean EWGPA by SAT Essay Reading Score 
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Figure 2 shows the average EWGPA earned by students at each Analysis score point. This 
figure shows a positive relationship between the Analysis score and first-semester EWGPA. 
As an example from the graph, you can see that the average EWGPA for a student with an 
Analysis score of 3 is a 3.30, while the average EWGPA for a student with an Analysis score 
of 7 is a3.72. 


Figure 2: Mean EWGPA by SAT Essay Analysis Score 
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Figure 3 shows the average EWGPA earned by students at each Writing score point. This 
figure also shows a positive relationship between Writing scores and first-semester EWGPA. 
As an example from the graph, you can see that the average EWGPA for a student with a 
Writing score of 3 is a 2.69, while the average EWGPA for a student with a Writing score of 
fis a3.63. 


Figure 3: Mean EWGPA by SAT Essay Writing Score 
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First-Year Grade Point Average (FYGPA) 


In addition to understanding the relationship between SAT Essay scores and EWGPA, we 
explored the relationship between each SAT Essay score and FYGPA (see Figures 4 
through 6). Figure 4 shows the average FYGPA that students earn by Reading score point. 
It is evident that as a student’s Reading score increases, so does the FYGPA earned in 
college. There is a clear positive relationship between Reading score and average FYGPA. 
As an example from the graph, the average FYGPA for a student with a Reading score of 3 
is a 2.74, while the average FYGPA for a student with a Reading score of 7 is a 3.45. The 
lowest score does show a slight deviation from this pattern, but this is likely due to the 
smaller and therefore less stable sample of students earning the lowest score value. 


Figure 4: Mean FYGPA by SAT Essay Reading Score 
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Figure 5 shows the average FYGPA that students earn by each Analysis score point. There 
is a rather clear positive relationship between Analysis score and average FYGPA. As an 
example, from the graph, the average FYGPA for a student with an Analysis score of 3 isa 
2.99, while the average FYGPA for a student with an Analysis score of 7 is a 3.52. The 
highest score of 8 does show a slight deviation from this pattern, but this is likely due to the 
smaller and therefore less stable sample of students earning the highest score value. 


Figure 5: Mean FYGPA by SAT Essay Analysis Score 
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Figure 6 shows the average FYGPA that students earn by each Writing score point. There is 
a rather clear positive relationship between Writing score and average FYGPA. As an 
example, from the graph, the average FYGPA for a student with a Writing score of 3isa 
2.51, while the average FYGPA for a student with a Writing score of 7 is a 3.44. The lowest 
score does show a slight deviation from this pattern, but this is likely due to the smaller and 
therefore less stable sample of students earning the lowest score value. 


Figure 6: Mean FYGPA by SAT Essay Writing Score 
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Discussion 


The aim of this study was to examine the relationship between operational SAT Essay 
scores and important related college outcomes, including first-semester EWGPA and 
FYGPA. Across the 10 institutions in this first-look study sample, a positive relationship is 
evident between all three SAT Essay score dimensions and both EWGPA and FYGPA. 


These findings suggest that the SAT Essay can be a useful tool in institutional admission, 
placement, and advisement processes that are focused on the level of writing preparedness 
with which a student is entering college. The findings also support the use of the SAT Essay 
scores to identify the kinds of writing-related instructional supports a student may need to be 
successful at an institution. 
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This study represents an initial look at operational SAT Essay score relationships with 
college outcomes and will be replicated on a much larger and diverse national sample when 
those data become available in spring 2019. Additional analyses examining relationships 
between SAT Essay scores and other scores on the SAT will be explored, as well as 
relevant institutional and student subgroup analyses. Analyses examining how the different 
SAT Essay score dimensions relate to different types of English or writing courses will also 
be examined. 


Given the importance of strong writing skills for both college and career readiness and 
success (Graham, Harris, & Herbert, 2011), a continued focus on research related to direct 
(e.g., essay) and indirect (e.g., multiple choice) writing assessment seems both practical and 
critically necessary. What may prove even more important is a deeper understanding of how 
admission officers, K-12 teachers, and college faculty can best use this information about 
students’ writing proficiency to help personalize instruction to strengthen their writing skills in 
both high school and college. 
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Appendix 


Example of SAT Essay Prompt and Source Text 


Prompt 


As you read the passage below, consider how Paul Bogard uses 


e evidence, such as facts or examples, to support claims. 

e reasoning to develop ideas and to connect claims and evidence. 

e Stylistic or persuasive elements, such as word choice or appeals to emotion, to add 
power to the ideas expressed. 


Adapted from Paul Bogard, “Let There Be Dark.” ©2012 by Los Angeles Times. 
Originally published December 21, 2012. 


At my family’s cabin on a Minnesota lake, | knew woods so dark that my hands disappeared 
before my eyes. | knew night skies in which meteors left smoky trails across sugary spreads 
of stars. But now, when 8 of 10 children born in the United States will never know a sky dark 
enough for the Milky Way, | worry we are rapidly losing night’s natural darkness before 
realizing its worth. This winter solstice, as we cheer the days’ gradual movement back 
toward light, let us also remember the irreplaceable value of darkness. 


All life evolved to the steady rhythm of bright days and dark nights. Today, though, when we 
feel the closeness of nightfall, we reach quickly for a light switch. And too little darkness, 
meaning too much artificial light at night, spells trouble for all. 


Already the World Health Organization classifies working the night shift as a probable 
human carcinogen, and the American Medical Association has voiced its unanimous support 
for “light pollution reduction efforts and glare reduction efforts at both the national and state 
levels.” Our bodies need darkness to produce the hormone melatonin, which keeps certain 
cancers from developing, and our bodies need darkness for sleep. Sleep disorders have 
been linked to diabetes, obesity, cardiovascular disease and depression, and recent 
research suggests one main cause of “short sleep” is “long light.” Whether we work at night 
or simply take our tablets, notebooks and smartphones to bed, there isn’t a place for this 
much artificial light in our lives. 


The rest of the world depends on darkness as well, including nocturnal and crepuscular 
species of birds, insects, mammals, fish and reptiles. Some examples are well known—the 
400 species of birds that migrate at night in North America, the sea turtles that come ashore 
to lay their eggs—and some are not, such as the bats that save American farmers billions in 
pest control and the moths that pollinate 80% of the world’s flora. Ecological light pollution is 
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like the bulldozer of the night, wrecking habitat and disrupting ecosystems several billion 
years in the making. Simply put, without darkness, Earth’s ecology would collapse... 


In today’s crowded, louder, more fast-paced world, night’s darkness can provide solitude, 
quiet and stillness, qualities increasingly in short supply. Every religious tradition has 
considered darkness invaluable for a soulful life, and the chance to witness the universe has 
inspired artists, philosophers and everyday stargazers since time began. In a world awash 
with electric light...how would Van Gogh have given the world his “Starry Night’? Who knows 
what this vision of the night sky might inspire in each of us, in our children or grandchildren? 


Yet all over the world, our nights are growing brighter. In the United States and Western 
Europe, the amount of light in the sky increases an average of about 6% every year. 
Computer images of the United States at night, based on NASA photographs, show that 
what was a very dark country as recently as the 1950s is now nearly covered with a blanket 
of light. Much of this light is wasted energy, which means wasted dollars. Those of us over 
35 are perhaps among the last generation to have known truly dark nights. Even the 
northern lake where | was lucky to spend my summers has seen its darkness diminish. 


It doesn’t have to be this way. Light pollution is readily within our ability to solve, using new 
lighting technologies and shielding existing lights. Already, many cities and towns across 
North America and Europe are changing to LED streetlights, which offer dramatic 
possibilities for controlling wasted light. Other communities are finding success with simply 
turning off portions of their public lighting after midnight. Even Paris, the famed “city of light,” 
which already turns off its monument lighting after 1 a.m., will this summer start to require its 
shops, offices and public buildings to turn off lights after 2 a.m. Though primarily designed to 
save energy, such reductions in light will also go far in addressing light pollution. But we will 
never truly address the problem of light pollution until we become aware of the irreplaceable 
value and beauty of the darkness we are losing. 


Write an essay in which you explain how Paul Bogard builds an argument to persuade his 
audience that natural darkness should be preserved. In your essay, analyze how Bogard 
uses one or more of the features in the directions that precede the passage (or features of 
your own choice) to strengthen the logic and persuasiveness of his argument. Be sure that 
your analysis focuses on the most relevant features of the passage. 


Your essay should not explain whether you agree with Bogard’s claims, but rather explain 
how Bogard builds an argument to persuade his audience. 
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